Sequence adaptive bit allocation for pictures encoding

ABSTRACT

A method and apparatus for encoding pictures of a moving pictures sequence according to an overall target bit-rate, such as in a MPEG video encoder. Each picture has an assigned picture coding type for which a quality factor is adaptively determined according to past bit usages, so that bits can be adaptively allocated amongst picture types for optimizing visual quality of the encoded pictures.

TECHNICAL FIELD

The present invention relates to a method and apparatus for encodingmoving pictures sequences. In particular, the present invention relatesto a method and apparatus for bit location in a video signal compressionsystem.

BACKGROUND ART

Methods for encoding moving pictures or video had been developed forefficient transmission and storage. A current art of such encodingmethods is found in MPEG2 Test Model 5, ISO/IEC JTC1/SC29/WG11/NO400,April 1993, and the disclosure of that document is hereby expresslyincorporated herein by reference. In this method, an input videosequence is organized into sequence layer, group-of-pictures, pictures,slices, macroblocks, and finally block layer. Each picture in agroup-of-pictures will be coded according to its determined picturecoding type.

The picture coding types used in the MPEG2 Test Model 5 includeintra-coded picture (I-picture), predictive-coded picture (P-picture),and bi-directionally predictive-coded picture (B-picture). TheI-pictures are used mainly for random access or scene update. TheP-pictures use forward motion predictive coding with reference topreviously coded I- or P-pictures (anchor pictures), and the B-picturesuse both forward and backward motion predictive/interpolative codingwith reference to previously coded I- or P-pictures. A group of pictures(GOP) is formed in encoded order starting with an I-picture and endingwith the picture before the next I-picture in the sequence.

A picture is partitioned into smaller and non-overlapping blocks ofpixel data called macroblocks (MB) before encoding. Each MB from a P- orB-picture is subjected to a motion estimation process in which forwardmotion vectors, and backward motion vectors for the case of a B-pictureMB, are determined using reference pictures from a frame buffer. Withthe determined motion vectors, motion compensation is performed wherethe intra- or inter-picture prediction mode of the MB is firstdetermined according to the accuracy of the motion vectors found,followed by generating the necessary predicted MB containing theprediction error.

The predicted MB is then subjected to discrete cosine transform (DCT)and quantization of the DCT coefficients based on quantization matricesand quantization step-size. The quantized DCT coefficients of the MB isthen run-length encoded with variable length codes (VLC) and multiplexedwith additional information such as selected motion vectors, MB codingmodes, quantization step-size, and/or picture and sequence information,to form the output bitstream.

Local decoding is performed by inverse quantizing the quantized DCTcoefficients, followed by inverse DCT, and motion compensation. Localdecoding is performed such that the reference pictures used in themotion compensation are identical to those used by any external decoder.

The quantization step-size (QS) used for quantizing the DCT coefficientsof each MB has a direct impact on the number of bits produced at theoutput of the run-length VLC encoding process, and therefore the averageoutput bit rate. It has also a direct impact on the encoding quality,which represents the output picture quality at the correspondingdecoder. In general, larger QS generates lower output bit rate and lowerencoding quality. In order to control output bit rate and picturequality so that the resulting bitstream can satisfy channel bandwidth orstorage limitation as well as quality requirements, rate control andquantization control algorithms are used.

Some methods for rate control and quantization control can be found inthe above mentioned MPEG-2 Test Model 5. These methods comprisegenerally a bit allocation process, a rate control process, and anadaptive quantization process. In the bit allocation process, a targetnumber of bits is assigned for a new picture to be coded according to anumber of previously determined and pre-set parameters. The rate controlstep then calculates a reference quantization step-size for each MBbased on the target bits for the picture and the number of bits alreadyused from the target bits in encoding MBs from that picture. In theadaptive quantization process, the calculated reference quantizationstep-size is scaled according to local activities of the MB, and anaverage MB activity determined from the current or a previously codedpicture. This scaling is done according to a level of masking effects ofcoding noise by human perception for MB with high or low activitieswithin a picture. A video buffer verifier (VBV) may also be employed insuch a way that underflow and overflow of the decoder input buffer areprevented as required by the MPEG standard to ensure a target bit rateis maintained.

It is assumed in the bit allocation process that the visual quality of acoded picture can be qualified with a single number V_(Q), expressed bythe formula: $\begin{matrix}{V_{Q} = \frac{K}{Q}} & (1)\end{matrix}$where Q is the average quantization step-size of the coded picture and Kis a constant quality factor which depends only on the picture codingtype. It is also assumed that the visual qualities of all encodedpictures should be maintained at a similar level within a GOP.Therefore, for all pictures within a GOP, the bit allocation processmaintains the following equality: $\begin{matrix}{\frac{K_{I}}{Q_{I}} \approx \frac{K_{P}}{Q_{P}} \approx \frac{K_{B}}{Q_{B}}} & (2)\end{matrix}$where Q_(I), Q_(P), Q_(B) are the respective average quantizationstep-sizes of coded I-, P-, and B-picture, and similarly K_(I), K_(P),K_(B) are respective pre-determined quality factors for I-, P-, andB-pictures. Although this equality does not apply to an entire picturessequence, it should be considered valid within a GOP as well as acrossconsecutive GOP boundaries. For simplicity, K_(I), of equation (2) isnormalized to the value of 1.

From the above assumptions, an equation for determining target bitallocation for a picture to be coded can be derived for each of thepicture coding types. The equations are given as follows for each of theI-, P- and B-picture coding type: $\begin{matrix}{T_{1} = \frac{R}{1 + {N_{P}\left( \frac{X_{P}K_{I}}{X_{I}K_{P}} \right)} + {N_{B}\left( \frac{X_{B}K_{I}}{X_{I}K_{B}} \right)}}} & (3) \\{T_{P} = \frac{R}{N_{P} + {N_{B}\left( \frac{X_{B}K_{P}}{X_{P}K_{B}} \right)}}} & (4) \\{T_{B} = \frac{R}{N_{B} + {N_{P}\left( \frac{X_{P}K_{B}}{X_{B}X_{P}} \right)}}} & (5)\end{matrix}$whereX_(I)=S_(I)Q_(I), X_(P)=S_(P)Q_(P), X_(B)=S_(B)Q_(B),and

-   -   S_(I), S_(P), S_(B) are number of bits used by previously        encoded I-, P-, B-picture respectively,    -   Q_(I), Q_(P), Q_(B) are the average quantization step-size used        by previously encoded I-, P-, B-picture respectively,    -   N_(P) and N_(B) are the number of P- and B-pictures remaining in        the current GOP with respect to the current picture to be coded,    -   R is the remaining number of bits assigned to the GOP according        to a target bit-rate, and    -   T_(I), T_(P), T_(B) are the calculated target bit allocation for        a new I-, P-, B-picture to be coded respectively.

An optional lower limit may be applied to the determined target bitallocation as given in the MPEG2 Test Model 5, hence: $\begin{matrix}{\left( {T_{I}\quad{or}\quad T_{P}\quad{or}\quad T_{B}} \right) \geq \frac{Bit\_ Rate}{K_{1} \times {Picture\_ Rate}}} & (6)\end{matrix}$where Bit_Rate is the target bitrate,

-   -   Picture_Rate is number of pictures coded per second, and    -   K_(I) is a constrant (eg. 8).

A typical video encoder system is designed to code picture sequenceswith various characteristics and complexities. In particular, sequenceswith little motion updates or complex motion scenes creates differentrequirements for coding pictures of difference picture coding types.

For example, a sequence with little motion updates may be best codedwith higher ratio of bits allocated to the anchor pictures (I-picturesand P-pictures) for visual quality improvements. On the other hand, asequence with complex motion scenes may be best coded with relativelyeven distribution of bits to pictures of all picture coding types formotion detail improvements, and hence higher ratio of bit allocation tothe B-pictures. Present systems based on fixed visual quality ratios,for example according to equation (2), do not adequately address thesechanges in sequence characteristics.

In other words, the pre-determined and fixed quality factors K_(I),K_(P), and K_(B) have more or less determined the ratios of the averagequantization step-sizes (ie. Q_(I), Q_(P), and Q_(B) ratios) to be usedfor coding pictures of different picture coding types. This relationshiplimits the adaptivity of bit allocation to different picture codingtypes of different motion characteristics.

Furthermore, the adaptivity of bit allocation should depend on thevisual quality itself. When higher visual quality is achieved for theanchor pictures, a feature which re-distributes the bits to B-picturesis desired such that both visual quality and motion details can bebalanced.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a method orapparatus for picture sequence compression which better caters forchanges in sequence and coding characteristics. In particular, an objectof the present invention is to provide a method or apparatus forimproving bit allocation.

While sophisticated methods of determining motion characteristics byframe difference detection or motion vector analysis may be employed,such methods may have significant impact on implementation cost.Implementation cost may be an important consideration for systems to bebroadly used. A further object of the present invention is therefore toprovide a method or apparatus for bit allocation with low implementationcomplexity.

One way of solving the above mentioned problems is to adaptivelydetermine new quality factors K_(I), K_(P), and K_(B) of equations (2),(3), (4), and (5) according to characteristics or parameters obtainedfrom past coded pictures from a sequence. This is in effect changing thetarget ratios of the average quantization step-sizes for I-, P-, andB-pictures such that visual quality and/or motion details can beimproved depending on the contents within the sequence.

A further simplified approach is to adaptively modify only the value ofK_(B), therefore the target ratios of the average quantization step-sizeof B-pictures over that of I- or P-pictures', as experimentallydetermined that the impact is not significant in terms of solving abovementioned problems when K_(I) or K_(P) are also modified adaptively.

The quality factors are modified adaptively in the present inventionaccording to the average quantization step-sizes and bit usage of pastcoded I-pictures, P-pictures and B-pictures. Encoding quality and codingcomplexity of a picture may be measured from its average quantizationstep-size and bits used. The ratios of coding complexities of picturesof different picture coding types together with encoding qualities maybe used to indicate spatial and/or motion complexity, and therefore, thequality factors may be determined accordingly.

A video encoding system encodes a video sequence of various picturecoding types (I-, P- and B-pictures) with sequence adaptive bitallocation according to the present invention. Initial values ofnecessary parameters for coding may be pre-determined. Bits areallocated to each picture to be coded such that the resulting visualqualities defined as K/Q of all pictures within a group-of-picturesremains at a similar level.

The quality factor K is dependent on the picture coding type; andfurthermore, K, is adaptively modified according to the averagequantization step-sizes (Q) and bit usages of past coded pictures.

With bits allocated to a picture to be coded, a rate controller is thenused to regulate bits for coding of each macroblock within the picture.A video encoding system according to the present invention may includecomponents that are similar to those used in the MPEG2 Test Model 5.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is described in greater detail hereinafter, by way ofexample only, with reference to a preferred embodiment thereof and theaccompanying drawings, wherein:

FIG. 1 is a block diagram illustrating a video encoding system withsequence adaptive bit allocation according to an embodiment of thepresent invention; and

FIG. 2 is a flow chart of an embodiment of a Quality Factor Estimator ofpresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 illustrates a video encoding system 100 with sequence adaptivebit allocation according to an embodiment of the present invention. Aninput video picture sequence is encoded by a video encoder 101 toproduce a compressed output bitstream which may be transmitted to anexternal decoder via a communication channel or recorded on digitalstorage media for playback applications. Each picture from the inputvideo sequence is coded according to its assigned picture coding type(I-, P-, or B-picture). A group-of-pictures (GOP) is formed startingwith an I-picture, and followed by possibly sets of P-pictures andB-pictures.

Typically, the video encoder 101 utilizes motion estimation, motioncompensation, discrete cosine transform (DCT) coding, andrun-length/differential encoding with variable length codes (VLC) asvideo compression techniques. A video encoder may be implemented suchthat the output bitstream is compliant to one of the ISO/IEC MPEGstandards, and can be decoded by the corresponding MPEG video decoders.

A bit allocation processor 103 is coupled to or forms part of the videoencoder, and is provided to determine a target number of bits (T) to beused for coding of a picture from the input video according to itspicture coding type and target bit-rate. The target bit-rate and picturecoding type are provided as input to the bit allocation processor 103.Bits are allocated so that the resulting visual qualities defined as K/Qof all pictures within a group-of-pictures remains at a similar level.The quality factor K depends on the picture coding type (eg. K_(I),K_(P), K_(B)) and Q is the average quantization step-size used forcoding that picture type. Example methods of bit allocation may be foundin MPEG2 Test Model 5, as given by equations (3), (4), (5) with anoptional lower limit given by (6). A set of initial values may beassumed for K_(I), K_(P), K_(B), X_(I), X_(P), and X_(B).

After determining the target bits (T), a bit rate controller 104 and anadaptive quantization processor 105 are used to calculate thequantization step-sizes for coding of each macroblock (MB) in thepicture. The quantization step-sizes are used optionally together withquantization matrices to quantize the MBs within the video encoder 101.The MBs may be subjected to motion estimation, motion compensation, anddiscrete cosine transform prior to quantization. Frame field adaptivemethods mentioned in MPEG2 Test Model 5 may also be applied. Thequantized MBs are run-length encoded with variable length codes andmultiplexed with all necessary side information to provide thecompressed output bitstream.

The rate controller 104 calculates a reference quantization step-sizefor each MB based on the target bits (T) for the pictures and the numberof bits consumed from the target bits in encoding MBs from that picture.For adaptive quantization 105, the calculated reference step-size isscaled according to local activities of the MB and an average MBactivity determined from the current or a previously coded picture.Methods of rate control and adaptive quantization may also be found inMPEG2 Test Model 5.

A quality factor estimator 102 is used to estimate the values of Kaccording to the picture coding type and bit usages (S) of past codedpictures supplied as input from the video encoder, and the averagequantization step-sizes (Q) which are supplied from the adaptivequantization processor 105. Together with the average quantizationstep-size Q, the quality factor K determines the visual quality of anencoded picture. A set of values for K may be determined for eachpicture coding type. Hence, K_(I), K_(P), and K_(B) may be adaptivelydetermined. Furthermore, the set of values for K may be updated at foreach picture or for every plurality of pictures, according tocomputation and visual quality adaptivity needs. The estimated values ofK are used by bit allocation processor 103 to determine the targetnumber of bits to be used for coding of each picture.

FIG. 2 illustrates a flow chart 200 of operational steps of the qualityfactor estimator according to an embodiment of the present invention.Starting with step 201, coding information such as the picture codingtypes, the average quantization step-sizes and bit usages of codedpictures are obtained. For step 202, encoding qualities are determinedusing the average quantization step-sizes. The encoding qualities forpast coded pictures of different picture coding types are determined forthe purpose of re-distributing bits to B-pictures when good encodingquality for P-pictures is achieved. Similarly, bits are re-distributedto I-pictures when good encoding qualities for P-pictures and B-picturesare achieved. For example, it is possible to set a maximum and minimumlimit for the value of K of a particular picture coding type usingaverage quantization step-sizes for past coded pictures as one suchmethod. In particular, a simplified embodiment may be given as follows:K_(B-MIN)=C_(O)K_(B-MAX)=C_(I)×Q_(P)+C₂, andC₃≦K_(B-MAX)≦C₄where C_(O) to C₄ are constants (eg. 1.4, 0.3, −0.4, 1.4, 2.0respectively),

-   -   Q_(P) is the average reference quantization step-size of a past        coded P-picture, and    -   K_(B-MAX) and K_(B-MIN) are the maximum and minimum quality        factor K for B-pictures.

Similarly, K_(I-MAX), K_(I-MIN), K_(P-MAX), and K_(P-MIN) for I-picturesand P-pictures may be determined. In fact, for simplicity K_(I-MAX),K_(I-MIN), K_(P-MAX), and K_(P-MIN) may also be normalized to theconstant value of 1.0.

At 203, the coding complexities (X_(I), X_(P), X_(B)) of past codedpictures are calculated. The coding complexity indicates spatial and/ormotion complexity in coding picture of a particular picture coding type.It is given as:X_(I)=S_(I)×Q_(I)X_(P)=S_(P)×Q_(P)X_(B)=S_(B)×Q_(B)where S_(I), S_(P), S_(B) are the number of bits generated by encoding apast I-picture, P-picture and B-picture respectively, and

-   -   Q_(I), Q_(P), Q_(B) are the average quantization step-size used        for encoding all macroblocks of the I-picture, P-picture and        B-picture respectively.

With the determined encoding qualities and coding complexities, thequality factors (K_(I), K_(P), and K_(B)) are estimated at the next step204. The ratios of coding complexity of different picture coding typeare used to define the ratios of the quality factors such that the bitallocation process can take into consideration spatial and/or motioncomplexities of different picture coding types. Furthermore, the ratiosof the quality factors are also limited according to the determinedencoding qualities so as to maximize visual quality and motion details.For simplicity, the quality factors can be normalized according to thefollowing example: $\begin{matrix}{{K_{I} = {{C_{5} \times \frac{X_{P}}{X_{I}}} + C_{6}}},{{{and}\quad{set}\quad K_{I - {MIN}}} \leq K_{I} \leq K_{I - {MAX}}}} & (7)\end{matrix}$K_(P)=1.0,  (8)$\begin{matrix}{{K_{B} = {{C_{7} \times \frac{X_{P}}{X_{B}}} + C_{8}}},{{{and}\quad{set}\quad K_{B - {MIN}}} \leq K_{B} \leq K_{B - {MAX}}}} & (9)\end{matrix}$where C₅ to C₈ are constants.

The values of C₅ to C₈ may be experimentally determined. It is alsopossible to fisher simplify the method by setting C₅ to C₈ to values of0, 1, 1, 0 respectively.

The bit allocation process attempts to maintain similar visual qualitywhich is based on the average quantization step-sizes and the qualityfactors (K_(I), K_(P), and K_(B)) while encoding pictures or groups ofpictures from an input video sequence. By adaptively modifying thequality factors according to picture coding types, and encodingqualities and coding complexities determined by the average quantizationstep-sizes and bit usages of coded pictures, a video encoder accordingto an embodiment of the present invention can efficiently allocate bitsfrom a target bit-rate to encode each pictures within the videosequence.

The quality factors are adapted in such a way that more bits are usedfor motion detail improvements when there are complex motion sceneswithin the picture sequence; and on the other hand, more bits areallocated to improve visual (spatial) quality when scenes within picturesequence contain little motion updates. The bit allocation is furtherchecked with the encoding qualities achieved so that the both motiondetails and visual quality can be optimized to enhance the overallobjective results of the video encoder.

1. A moving pictures encoder for encoding pictures in a moving picturessequence according to an overall target bit-rate, each picture having anassigned picture coding type, the moving pictures encoder including: aquality factor estimator including means for determining a picturecomplexity for each assigned picture coding type based on a number ofbits used for past coded pictures of that type, and an averagequantization step-size for past coded pictures of that type; a bitallocation processor for determining a target number of bits for codinga picture according to the overall target bit-rate, the assigned picturecoding type and a quality factor for the assigned picture coding type,the quality factor being defined as the product of a visual quality andthe average quantization step-size for past coded pictures of that type;a bit-rate controller for determining a reference quantization step-sizebased on the target number of bits and the number of bits used for thepicture; and an adaptive quantization processor for determining a codingquantization step-size based on the reference quantization step-size andpicture activities for the picture, and for determining said averagequantization step-size for past coded pictures, wherein the qualityfactor estimator includes means for determining the quality factor ofthe assigned picture coding type as a function of a ratio of the picturecomplexity of the assigned picture coding type to the picture complexityof at least one other picture coding type.
 2. A moving pictures encoderas claimed in claim 1 wherein a corresponding quality factor isdetermined by the quality factor estimator for each picture type.
 3. Amoving pictures encoder as claimed in claim 2 wherein a value for thequality factor is determined for coding of each picture of thecorresponding picture type.
 4. A moving pictures encoder as claimed inclaim 2 wherein a value for the quality factor is determined for codingof each plurality of pictures of the corresponding picture type.
 5. Amoving pictures encoder as claimed in claim 2, wherein minimum andmaximum limits are set for the quality factor of each picture type.
 6. Amethod for encoding pictures in a moving pictures sequence according toan overall target bit-rate, each picture having an assigned picturecoding type, including the steps of: determining a picture complexityfor the assigned picture coding type based on a number of bits used forpast coded pictures of that type, and an average quantization step-sizefor past coded pictures; determining a target number of bits for codinga picture according to the overall target bit-rate, the assigned picturecoding type, and a quality factor for the assigned picture coding type;determining a reference quantization step-size based on the targetnumber of bits and the number of bits used for the picture; determininga coding quantization step-size based on the reference quantizationstep-size and picture activities for the picture, and the averagequantization step-size for past coded pictures; and determining thequality factor of the assigned picture coding type as a function of thepicture complexities of the assigned picture coding type and of at leastone other picture coding type.
 7. A method as claimed in claim 6,wherein a corresponding quality factor is determined for each picturetype.
 8. A method as claimed in claim 7, wherein a value for the qualityfactor is determined for coding of each picture of the correspondingpicture type.
 9. A method as claimed in claim 7, wherein a value for thequality factor is determined for coding of each plurality of pictures ofthe corresponding picture type.
 10. A method as claimed in claim 7,wherein minimum and maximum limits are set for the quality factor ofeach picture type.
 11. An encoder for encoding pictures in a movingpictures sequence according to an overall target bit-rate, each picturehaving an assigned picture coding type, the encoder comprising: aquality factor estimator including means for determining a picturecomplexity for each picture coding type based on a number of bits usedfor past coded pictures of that type, and an average quantizationstep-size for past coded pictures of that type; a bit allocationprocessor for determining a target number of bits for coding theselected picture according to the overall target bit-rate, the picturecoding type and a quality factor for the assigned picture coding type,the quality factor being defined as the product of a visual quality andthe average quantization step-size for past coded pictures of that type;a bit-rate controller for determining a reference quantization step-sizebased on the target number of bits and the number of bits used for thepicture; and an adaptive quantization processor for determining a codingquantization step-size based on the reference quantization step-size andpicture activities for the picture, and for determining said averagequantization step-size for past coded pictures, wherein the qualityfactor estimator includes means for determining the quality factor ofthe assigned picture coding type as a function of a ratio of the picturecomplexity of the assigned picture coding type to the picture complexityof at least one other picture coding type.
 12. The encoder of claim 11wherein the quality factor estimator determines a quality factor for theselected picture based on the number of bits used for past codedpictures of the picture coding type of the selected picture.
 13. Theencoder of claim 11 wherein the quality factor estimator determines aquality factor for the selected picture based on the number of bits usedfor past coded pictures of a different picture coding type than theassigned picture coding type of the selected picture.
 14. The encoder ofclaim 11 wherein the quality factor estimator further determines thequality factor for the selected picture based on quality factors forpast coded pictures of a different picture coding type than the picturecoding type of the selected picture.
 15. The encoder of claim 11 whereinthe quality factor estimator determines the quality factor for theselected picture within predetermined minimum and maximum limits.
 16. Amethod for encoding pictures in a moving pictures sequence, each picturehaving an assigned picture coding type assigned from a predeterminedselection of picture coding types, the method comprising: dynamicallycalculating a quality factor for a selected picture having a firstassigned picture coding type as a function of a ration of a complexitycalculation for the first assigned picture coding type to a complexitycalculation for a second assigned picture coding type other than thefirst assigned picture coding type so that the quality factor for theselected picture matches the quality factors for the past coded pictureswithin a predetermined range, the complexity calculation for each of thefirst and second assigned picture coding types being based on a numberof bits generated by encoding a past picture of the same assignedpicture coding type; determining a target number of bits for coding theselected picture according to the overall target bit-rate, the firstassigned picture coding type, and the calculated quality factor;determining a coding quantization step-size based on the target numberof bits; and encoding the selected picture according to the determinedcoding quantization step-size.
 17. The method of claim 16 wherein thequality factor for the selected picture is further based on a number ofbits used for past coded pictures, and an average quantization step-sizefor past coded pictures.
 18. The method of claim 16 wherein thequantization step-size is determined based on the reference quantizationstep-size and picture activities for the selected picture, and anaverage quantization step-size for past coded pictures.
 19. The methodof claim 16 wherein the predetermined range is used to calculate aminimum and maximum quality factor values for the selected picture. 20.The method of claim 16 wherein the quality factor for the selectedpicture is calculated based on the number of bits used for past codedpictures of the first picture coding type.
 21. The method of claim 16wherein the quality factor for the selected picture is calculated basedon the number of bits used for past coded pictures having a picturecoding type different from first picture coding type.